Skip to content

Use async HTTP requests for UI and stats (#835)#845

Closed
Eclipse1982 wants to merge 1 commit into
jdolan:mainfrom
Eclipse1982:feature/async-ui-http
Closed

Use async HTTP requests for UI and stats (#835)#845
Eclipse1982 wants to merge 1 commit into
jdolan:mainfrom
Eclipse1982:feature/async-ui-http

Conversation

@Eclipse1982

@Eclipse1982 Eclipse1982 commented Jun 12, 2026

Copy link
Copy Markdown
Contributor

Summary

Replaces the blocking HTTP requests in the UI / stats code paths with asynchronous requests, so the client no longer shows a black screen at launch (GUID hash fetch) or freezes when opening the Leaderboard and Stats menus while waiting on the stats API.

Completion callbacks run on the HTTP session thread; results are staged and marshaled to the main thread via MVC_NOTIFICATION_EVENT, and view controllers refresh in respondToEvent — following the pattern suggested in the issue and already used by NOTIFICATION_SERVER_PARSED.

Fixes #835

Changes

  • Added Net_HttpGetInstanceAsync / Net_HttpGetInstancesAsync — async counterparts to the JSON instance helpers — and exposed them through the cgame import (cgi.HttpGetInstanceAsync, cgi.HttpGetInstancesAsync).
  • Cl_InitGuidHash now fires an async request at startup; the hash is applied to guid_hashed on the main thread via a new NOTIFICATION_GUID_HASHED event, so startup never blocks on the network. The call was moved after Ui_Init, because MVC_NOTIFICATION_EVENT is only registered with SDL once ObjectivelyMVC initializes — a completion firing earlier would push event type 0 and the hash would be silently dropped.
  • LeaderboardViewController fetches asynchronously (initial load and sort-column clicks); rows are staged under a mutex and applied in respondToEvent on NOTIFICATION_LEADERBOARD_FETCHED. It also reloads when the GUID hash arrives so the local player's row highlights.
  • StatsViewController fetches asynchronously, showing placeholders until NOTIFICATION_STATS_FETCHED delivers the response; it refreshes when NOTIFICATION_GUID_HASHED arrives (previously a cold start could show "Sign in" until the next visit).
  • Generation counters discard stale responses when a newer fetch supersedes an in-flight one; callbacks never touch UI state directly, so there are no dangling view controller pointers.
  • Added check_http unit tests covering both new async helpers against the local test HTTP server.

Testing

  • Builds without warnings (make -j$(nproc))
  • Tests pass (make check)
  • Tested in-game on Linux (Debian 12)

Notes

  • Built and tested on Debian 12 with the from-source dependency stack from linux/docker/Dockerfile.build. The changed files compile without warnings (pre-existing warnings elsewhere in the tree are untouched).
  • make check: 18/19 suites pass, including check_http with the two new async tests. The one failure is check_filesystem ("Failed to load quetoo.cfg") — an environment issue in the build container (no game data installed), unrelated to this change.
  • In-game smoke test against the live stats API (giblets.quetoo.org), headless under Xvfb/llvmpipe: client reaches the main menu with stats placeholders immediately, the GUID hash arrives asynchronously and the Stats tab populates, the Leaderboard tab loads live rows, sort-column clicks refetch and reorder without any UI stall, and selectOwnRow highlights the local player after the async reload. No warnings or errors in the console log.

Replace blocking HTTP calls in menu code paths with asynchronous
requests so the client no longer freezes (or shows a black screen at
launch) while waiting on the stats API:

- Add Net_HttpGetInstanceAsync and Net_HttpGetInstancesAsync, async
  counterparts to the JSON instance helpers, and expose them through
  the cgame import.
- Fetch the GUID hash asynchronously at startup; the result is applied
  on the main thread via a NOTIFICATION_GUID_HASHED MVC event.
- Fetch the leaderboard asynchronously; completed results are staged
  and applied in respondToEvent on NOTIFICATION_LEADERBOARD_FETCHED.
- Fetch player stats asynchronously; completed responses are staged
  and applied in respondToEvent on NOTIFICATION_STATS_FETCHED. Stats
  also refresh when the GUID hash arrives.
- Generation counters discard stale responses when a newer fetch has
  been issued; completion callbacks never touch UI state directly.
- Add check_http unit tests covering both new async JSON helpers.
@Eclipse1982 Eclipse1982 force-pushed the feature/async-ui-http branch from c84a48e to ef9d838 Compare June 12, 2026 23:05
@jdolan

jdolan commented Jun 13, 2026

Copy link
Copy Markdown
Owner

Fantastic! Thank you. I'm working on some improvements to the JSON deserialization code here first. I'll commit that and up-merge into your branch and get this over the finish line. This is a big win for user experience.

jdolan added a commit that referenced this pull request Jun 15, 2026
Port the async HTTP UI pattern from PR #845, rewritten on top of RESTClient:

- cl_main: Cl_InitGuidHash fires RESTClient::getAsync; completion parses
  JSON on the HTTP thread and pushes NOTIFICATION_GUID_HASHED with the
  hashed GUID as SDL event data1. Cl_GuidHashedEvent applies it on the
  main thread. Moved after Ui_Init so the event type is registered.
- cl_input: Routes NOTIFICATION_GUID_HASHED to Cl_GuidHashedEvent before
  forwarding to UI and cgame.
- StatsViewController: fetchStats fires getAsync; fetchStatsComplete
  retains Data and pushes NOTIFICATION_STATS_FETCHED. respondToEvent
  picks it up, calls loadStats with JSONContext on the main thread.
  Also re-fetches on NOTIFICATION_GUID_HASHED.
- LeaderboardViewController: same pattern with NOTIFICATION_LEADERBOARD_FETCHED.
  respondToEvent parses Data with JSONContext, reloads table, selects own row.
  Also reloads on NOTIFICATION_GUID_HASHED.
- cl_types: Add NOTIFICATION_GUID_HASHED, _LEADERBOARD_FETCHED, _STATS_FETCHED.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@jdolan

jdolan commented Jun 16, 2026

Copy link
Copy Markdown
Owner

Closing in favor of #851 which borrowed from this.

@jdolan jdolan closed this Jun 16, 2026
jdolan added a commit that referenced this pull request Jun 16, 2026
…851)

* Default stats/leaderboard period to this week (#836)

Add a period Select control to both StatsViewController and
LeaderboardViewController with options This Week / This Month /
This Year / All Time, defaulting to This Week.  The selected period
is translated to from/to date parameters on every API request so
the leaderboard and personal stats only show the current week's
activity by default.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* Migrate quetoo HTTP/JSON layer to JSONContext API

Replace all uses of the old JSONSerialization / JSONProperty API with the
new instance-based JSONContext / JSONProperties API:

- net_http: Net_HttpGetInstance/Instances take const JSONProperties *
  instead of const JSONProperty *; stride param removed from
  Net_HttpGetInstances (size comes from properties->size)
- cgame.h: HttpGetInstance/Instances function pointer signatures updated
  to match
- sv_game.c: Sv_PostStats uses JSONContext + MakeJSONCharactersProperty /
  MakeJSONBooleProperty / MakeJSONInt32Property; properties hoisted to
  file scope to satisfy static-initializer requirements
- cl_main.c: guid_hash_properties converted to JSONProperties struct
- LeaderboardViewController: leaderboard_properties converted; stride
  removed from HttpGetInstances call
- StatsViewController: removed broken parseNemesis/parseKillsByWeapon
  callbacks; replaced with static property descriptors and
  MakeJSONObjectProperty / MakeJSONArrayProperty
- check_http.c: all property arrays converted to JSONProperties structs;
  http_parse_owner/items callbacks replaced with MakeJSONObjectProperty /
  MakeJSONArrayProperty; call sites updated (no stride)

All 19 tests pass.

Replace MakeJSON*Property convenience macros with explicit MakeJSONProperty

Now that Objectively has removed the convenience macros, call sites
name serializer/deserializer pairs and data arguments directly.  The
explicit form makes the type binding transparent and symmetric with
the rest of the MakeJSONProperty usage in the codebase.

19/19 tests pass.

StatsViewController cleanup.

Ignore check_net_message executable.

Co-Authored-By: Copilot <223556219+Copilot@users.noreply.github.com>

* Rename instance→struct in JSON HTTP API

- Net_HttpGetStruct / Net_HttpGetStructs
- HttpGetStruct / HttpGetStructs in cgame import
- JSONSerializeStruct / JSONDeserializeStruct at all call sites
- structFromData / structsFromData in net_http.c

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* NULL optimization in JSON property descriptors

- sv_game.c: NULL deserializers (serialize-only)
- cl_main.c, LeaderboardViewController.c: NULL serializers (deserialize-only)
- StatsViewController.c: NULL serializers; consolidate nemesis/kills_by_weapon
  into file-scope statics tied into a single stats_properties declaration
- check_http.c: NULL serializers; consolidate nested response properties
  into file-scope statics

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Remove JSONFieldSize from all MakeJSONProperty call sites

Now that JSONProperty carries .size automatically, all JSONFieldSize(...)
arguments become NULL, which also frees data for custom deserializer use.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Update JSONArrayProperties field names: count→capacity, count_offset→count

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Cosmetics.

* LeaderboardViewController: remove period Select, hard-code current month

- Remove periodSelect widget, static period strings, didSelectPeriod delegate
- fetchLeaderboard now always uses periodDateRange("month", ...)
- Remove leaderboardControls StackView from JSON layout
- Remove .leaderboardControls CSS
- StatsViewController: append ?from=YYYY-MM-01&to=YYYY-MM-DD for current month

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Replace Net_Http* wrappers with RESTClient across the codebase

- Remove net_http.{c,h}; add net_http_server.{c,h} with only
  server-side utilities (Net_HttpUrl, Net_HttpParseRequestLine,
  Net_HttpFormatResponse, Net_HttpSendError)
- Replace Http{Get,GetStruct,GetStructs,GetAsync} fields in cgi_t
  with a single RESTClient *http field; wire via sharedInstance in
  cl_cgame.c with URLCache configured on first load
- Update all callers to use RESTClient directly:
  cl_main, cl_parse, installer, sv_game, master/main
- cgame callers (StatsViewController, LeaderboardViewController,
  UpdateViewController) use cgi.http + JSONContext
- cl_main cache clear uses URLCache::removeAllCachedResponses
- check_http: remove Net_HttpGet* tests; roundtrip uses RESTClient

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* s/http/restClient/g

* Make GUID hash, stats, and leaderboard HTTP fetches async (#845)

Port the async HTTP UI pattern from PR #845, rewritten on top of RESTClient:

- cl_main: Cl_InitGuidHash fires RESTClient::getAsync; completion parses
  JSON on the HTTP thread and pushes NOTIFICATION_GUID_HASHED with the
  hashed GUID as SDL event data1. Cl_GuidHashedEvent applies it on the
  main thread. Moved after Ui_Init so the event type is registered.
- cl_input: Routes NOTIFICATION_GUID_HASHED to Cl_GuidHashedEvent before
  forwarding to UI and cgame.
- StatsViewController: fetchStats fires getAsync; fetchStatsComplete
  retains Data and pushes NOTIFICATION_STATS_FETCHED. respondToEvent
  picks it up, calls loadStats with JSONContext on the main thread.
  Also re-fetches on NOTIFICATION_GUID_HASHED.
- LeaderboardViewController: same pattern with NOTIFICATION_LEADERBOARD_FETCHED.
  respondToEvent parses Data with JSONContext, reloads table, selects own row.
  Also reloads on NOTIFICATION_GUID_HASHED.
- cl_types: Add NOTIFICATION_GUID_HASHED, _LEADERBOARD_FETCHED, _STATS_FETCHED.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Use static staging buffers for async HTTP results; no heap, no mutex

Replace the mutex+Data* and generation counter approach with simple
static staging buffers:

- stats_pending (StatsResponse): hydrated by fetchStatsComplete on the
  HTTP thread; respondToEvent copies it into this->stats on the main thread.
- leaderboard_pending[]/leaderboard_pending_count: hydrated by
  fetchLeaderboardComplete; respondToEvent memcpys the entries.

The SDL event queue provides the happens-before guarantee between the
write (before SDL_PushEvent) and the read (after SDL_PollEvent), so no
mutex is required. No heap allocation means no ownership transfer and
no possible leak regardless of ViewController lifecycle.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Introduce LeaderboardResponse to encapsulate the leaderboard array and count

Mirrors StatsResponse. LeaderboardResponse{entries[], num_entries} defined
in the header; the static staging buffer and the ViewController field both
use it. Deserialization still goes through structsFromData (the API returns
a bare JSON array) into leaderboard_response.entries.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Remove client-side date formatting; fix loadStats status gate

The API now defaults to the current calendar month when from/to are
omitted, so clients no longer need to compute or send date parameters.

- Remove periodDateRange() and <time.h> from LeaderboardViewController
- Remove strftime/localtime date block from StatsViewController::fetchStats
- Add pendingStatsStatus alongside pendingStatsResponse so loadStats
  gates on HTTP 200 rather than non-zero struct fields, fixing the
  false 'Error' shown for players with no activity this month

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Make GUID fetching synchronous.

* sv_game: build frag/capture JSONProperties as locals to fix GCC 11 build (#852)

GCC 11 (ubuntu-22.04, the build-linux CI runner) rejects the file-scope
`static const JSONProperties` descriptors initialized via MakeJSONProperties
with "initializer element is not constant": the macro expands to a compound
literal whose nested (const JSONProperty[]){...} is built from per-field
compound literals, which GCC 11 does not treat as an address constant in a
static initializer. Newer GCC and MSVC accept it, so build-windows stayed
green.

Move both descriptors into Sv_PostStats as automatic `const JSONProperties`
locals, scoped to the frags/captures branches that use them. Automatic
objects permit non-constant initializers, so all toolchains compile. Cold
POST-path code, so per-call construction is negligible.

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>

* Simplify guid fetching and stats loading.

* Whitespace.

* Bump cgame version.

* Parse captures from stats API

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* Minor PR feedback.

* Simplify guid_hash handling.

* Fix GCC static initializer errors in StatsViewController and LeaderboardViewController

GCC rejects compound literals (from MakeJSONProperties macro) as
initializers for file-scope static variables. Split each MakeJSONProperties
call into a named static JSONProperty array and a separate JSONProperties
struct, giving GCC a proper address-constant for the .properties pointer.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Fix -Wunused-result and -Wenum-conversion warnings on Linux/GCC

- sys.c: cast return values of system() and write() to void
- voxel.c: add (mem_tag_t) casts for quemap anonymous enum tags,
  matching the existing pattern on lines 185 and 665

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: James Harrington <85718676+Eclipse1982@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Use async HTTP requests for UI / stats where possible.

2 participants